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Abstract 

In this paper we study random linear systems with k variables per 
equation over the finite field GF{2), or equivalently fc-XOR-CNF formulas. 
In a previous paper Creignou and Daude proved that the phase transition 
for the consistency (satisfiability) of such systems (formulas) exhibits a 
sharp threshold. Here we prove that the phase transition occurs as the 
number of equations (clauses) is proportional to the number of variables. 
For any > 3 we establish first estimates for the critical ratio. For 
fc = 3 we get 0.93 as an upper bound, 0.89 as a lower bound, whereas 
experiments suggest that the critical ratio is approximately 0.92. 
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1 Introduction 



For fc-CNF formulas many experiments have shown a very swift transition be- 
tween satisfiabihty and unsatisfiabihty as the ratio L/n of the number of clauses 
over the number of variables is varied. That is, there exists Cfc, a critical value of 
L/n , such that li L/n < Ck then the formula is almost surely satisfiable and if 
ii L/n > Ck then the formula is almost surely unsatisfiable. Most of the papers 
investigating this phase transition are directed towards obtaining approximate 
estimates of its location. For instance for 3-SAT, for an observed sharp thresh- 
old of about C3 = 4.25 the best lower bound is 3.003 1^ and tight upper bounds 
have been successively obtained, 4.601 4.596 |] and finally 4.506 ||]. In 
1999, Friedgut proposed a new and fruitful approach. In a remarkable paper 

with an appendix by Bourgain, he developed a general sharp threshold cri- 
terion for monotone subsets of the hypercube. In using this criterion he proved 
that fc-SAT and (in collaboration with Achlioptas) fc-col Q exhibit a sharp 
threshold. In Creignou and Daude applied this criterion to the fc-XOR-SAT 
problem (in which the usual "or" is replaced by the "exclusive or"). Thus, they 
proved the existence of a sharp threshold phenomenon for fc-XOR-SAT, fc > 3, 
without specifying its location. 

The aim of this paper is to prove that the phase transition for fc-XOR-SAT 
occurs as the number of clauses is proportional to the number of variables and 
more precisely to provide approximate estimates of its location. In Section^ we 
give a lower bound and in Section ^ an upper bound. Our results rely first on 
the sharpness of the threshold established in Q (see Section ||) and second on 
the handy translation of our problem in terms of random matrices (see Section 
m . These theoretical results are supplemented with experiments in Section o. 



2 Notation and definitions 

Throughout the paper fc will denote an integer equal to or greater than 3. 
A k-XOR-clause (or shortly a k-equation), C, is a linear equation over the 
finite field GF{2) using exactly fc variables, C = {{xi ® . . . ® xu) = e) where 
e = or 1. A k-XOR-formula (or shortly a k-system) is a conjunction of distinct 
fc-XOR-clauses. A truth assignment / is a mapping that assigns or 1 to each 

variable in its domain, it satisfies an XOR-clause C — {{xi . . . ® Xk) = e) iff 

p 

I{C) :— ^^/(xi) mod 2 — e and it satisfies a formula F iff it satisfies every 

i=l 

clause in F. 

We will denote by fc-XOR-SAT (or shortly SAT) the property for a fc-XOR- 
formula of being satisfiable (or equivalently the property for a fc-system of being 
consistent) and by UNSAT the property of being unsatisfiable. The property 
UNSAT is monotone increasing. 

Throughout the paper we reserve n for the number of variables {{xi, . . . , Xn} 
denotes the set of variables). There are Nk = 2(^) different fc-XOR-clauses over 
n variables. We consider the random formula obtained by choosing uniformly. 



2 



independently and with replacement L clauses from the A^^ possible fc-clauses. 
This defines a probability space of fc-XOR-formulas consisting of all ordered 
sets of L clauses, not necessarily distinct, with k distinct variables, each over 
a set of Boolean variables, each set of clauses having the same probability. 
This probability space is denoted by VL{n, L, k), the associated probability is the 
uniform law: 

ysen{n,L,k) p„,i(s) = 

We are interested in estimating the probability that a formula drawn at random 
from Vl{n,L,k) is satisfiable, that is in estimating P„.L(fc-XOR-SAT). 

Our model for producing a random formula relates to the model in which 
each of the possible clauses is chosen independently with probability p in the 
same way G{n, L) relates to G{n,p) in random graph theory. In most investi- 
gations on properties of random subgraphs of the complete graph on n vertices, 
these two models are practically interchangeable, provided the number, L, of 
edges be close to p[^) (see [^). In the first two authors proved that when 
each clauses is chosen with probability p then fc-XOR-SAT exhibits a sharp 
threshold for fc > 3. They also noted that the transition occurs when pNk < n. 
Thus in the context of random formula these results are still valid in our model 
( provided p ^ L/N^), they can be expressed as: 

Theorem 2.1 [g 

For every fc > 3 there exists a function Ck (n) < 1 such that: for every e > 
^\im^Pn,ic,in)-e)n{k-XOR-SAT) = 1 

"ljm%n,(c.(«)+e)„(fc-^Oi?-5^7) = 

We are going to prove that this sharp threshold behavior occurs as the num- 
ber of clauses is proportional to the number of variables of formulas and more 
precisely we are goin g to provide lower and upper bounds for Cfc(n). Since we 
know from Theorem that Ck{n) < 1, we can and we will suppose in the 
sequel that L < n. 

As mentioned in Q , though there is a swift transition of probability of satisfi- 
ability it is still feasible that the critical value Cfc(n) does not converge to any 
given value. However, one can define (3k = limsupcfc(n) and — liminf Ck{n). 

Thus, Pk (respectively, ak) is the least (greatest) real number such that ii c > Pk 
(c < ak) then the probability of a fc-XOR- formula with n variables and cn 
clauses being satisfiable converges to (to 1) as n tends to infinity; hence 
ttfc < Pk and experiments suggest strongly (see Section ||) that equality holds. 

In a first step (Section ^ we will reduce our problem to the study of the 
rank of random sparse matrices over GF{2). Then, we will give a lower bound 
(Section H) and an upper bound (Section g) for the threshold. 

At last, H{x) will denote the well-known entropy function, 

H{x) = X \n{x) + (1 - a;) ln(l - x). 
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3 From random formulas to random matrices 



The aim of this section is to express P„^L(fc-XOR-SAT) in terms of random 
matrices. We will first use a technique related to the harmonic mean formula 
and publicized in and This technique provides a simple expression for 
the number of satisfiable systems and can be described by a bipartite graph G 
formed by {0, 1}" (the set of all assignments / : {a:i, . . . ,a;„} — > {0, 1}) as 
the first part of vertices and Vl{n, L, k) (the set of all systems s) as the second 
part. There is an edge {/, s} in G if the assignment / satisfies the system s. 
The degree of every assignment / is equal to (^) and for every system s the 
degree d{s) of s is the number of distinct assignments satisfying s. The number 
of satisfiable fc-systems is thus given by 

|fc-XOR-SAT|= Yl E rfiy- 
/e{o,i}" s,i(s)=i ^ ' 

As there is a one-to-one correspondence from the set of linear systems satisfied 
by some fixed assignment /q onto the set 7i of homogeneous linear systems, we 
get: 

Ifc-XOR-SATI 2" y -i-. 

^ dis) 
sen ^ ' 

From this simple expression we get the following theorem. 

Theorem 3.1 Let Yi^A) denote the number of vectors in the kernel of the trans- 
pose of A, then providing A be chosen uniformly in the set ML,n.k of Boolean 
L X n matrices with exactly k units in each row: 

L 

Pn,L{k-XOR-SAT) = ^2'-^P(,) = ElM^/Y) 

where P^^) is the probability that a matrix from ML^„,k is of rank r, and k 
denotes the expectation. 

Proof: From our preliminary work we have: 

|fc-XOR-SAT| = 2" Y: 2-r'-HA) = E E 2^ 

AeML „ fc r=0 AeML,„,k 

Tar,k{A)=r 

But #{A e ML,n,k I rank{A) = r} = \ML,-a,k\P{r) and 
P„,L(fc-XOR-SAT) = |A:-XOR-SAT|/ (2(^))^, as |Mi,„,fe| = (^)'' we get: 



P„,z,(fc-XOR-SAT) = ^2'--^F( 



r=0 



The second equality is justified by the following fact: 
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{rank{A) ^ r) ^ Y = 2^"^ 

I 



We will see that this result leads to two key facts, Proposition 4.1 and 



Proposition 5.1, which enable us to give tight bounds for the threshold. 



4 A lower bound for the threshold 

By Theorem and Jensen's inequality we have: 

P„.l(A:-XOR-SAT) = ElA'^/Y) > \ (1) 
Thus, we get the first key fact: 

Proposition 4.1 // \imEcn.k{Y) = 1 then at > c. 

n ' 

A first lower bound for the threshold of fc-XOR-SAT can be derived from 
the following result 

Theorem 4.2 Let 0j^. denote the minimum over [0, 1/2] of the function 

^_ \n2 + H{x) 
' " ln(l + (1 - 2.x)fc) 

then for any c < 9k, lim Ecn,kiY) = 1 

From maple's estimates for 9k we get the following bounds: 

as > 0.88949, 0:4 > 0.96714, ag > 0.98916, ae > 0.99622 (2) 



To prove Theorem 4.2 we first show that the behavior of EL,k{Y) is given 

[uantity: 

fe 



by the one of the following quantity 

fe 



s=0 



m \ I n — m 
s I \ k — s 



Proposition 4.3 ElmO") = 2"" ^ ( " ) 



m=0 ^ ' \k) 



Proof: Let us consider X{A) = \ kerA|, X is the sum of indicator variables: 

X — Xfi where Xfi{A) = 1 if and only if u e kerA. 

«e{o,i}" 
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By symmetry, E[X) — ( where Um is any vector of weight m. 

m=o 

For each Um, consider the subset /g^ of {1, • • • , n} formed by the indexes of the 
unit coordinates of Um- Then Um is in the kernel of A if and only if each row of 
the submatrix of A formed by the m columns whose indexes are in I^^ has an 
even number s of unit coefficients. Thus, we have 



E 



ra \ I n — 7n 
s ) \ k ~ s 



s— mod 2 

possible rows for a matrix A in Mnx.k such that Xfi^{A) — 1, and we deduce: 



L 

n 



\s=0 mod 2 ^ ^ ^ ^ / 



therefore, i?(X) = 2-- ;^ r (l + ^)^ 

As y = 2^^"X, the conclusion follows. I 



This result shows that if we split E]^ k{Y) in two then the proof of Theorem 



4.2 follows from the two following claims: 



Claim 4.4 If L < n then limSi(L, k) — 1, where 



|m— ■^|<n 3fc 

Claim 4.5 If c < Ok then \im. Tj 2 (On^k) — 0^ where 



'n.m,k 



\m-^\>n 3k 

These claims rely on asymptotical properties of sums of binomial coefficients 
and of the quantity 0Jn,m.,k- Observe that for each fc, ujn,m,k can be viewed as a 
polynomial over two variables n and m of total degree k, this is made precise 
by the following: 

Proposition 4.6 

(n — 2m)'' n(n ~ 2m)'^^^ 
= T 2(fc,2)! +^^("'"^) 

where Pk is a polynomial on two variables of total degree fc — 2. Thus, there 
exists absolute positive constants Ak and such that: 
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<^ 


1 - 


2m 


n 


n 




n 



k-2 Bk 



Proof: Let us show how we get the leadmg term. Observe that: 

k t: / \k — s 



(3) 



. to" (n — to) 



+ P{n, to), 



where P{n, to) is a polynomial of degree /c — 1. But, 



. to" (n — to) 



fe-s 



=1 



fc! 



s=0 ^ ' s=0 

The study of the second term is left to the reader. 



I 



^Fromthis result and from its following consequences, Lemma 4.7 and 
Lemma 4.8, it turns out that that the asymptotical behavior of Yi\(L^W) and 



E2(i, k) is given by the one of (1 + (1 ) 

n 

Lemma 4.7 If L < n and |m — ^| < n^^sr then 

n 

Proof: From (|^) and from the mean value theorem we have when L < 



(;:) 



'_(! + (!__)* 
n 







k-2 






^ 2m 


H 




{^+ln,m,kj 




n 




n J 



L-1 



with 7„,m,fc lying between 



and (1 - ^)^ 
2to ,,, 



fc-2 



0{n — ). Thus, (|) gives 



As|to— 2.|<jji 3fc and fc > 3 we get 11 1 

n — 

y^'"" - (1 = 0(n~5^) which shows that = 0{n^). Since 

ik) ^ (fe) 

we also have |1 1*^ — 0{n~), we can conclude that m k are 0{n~). 

n ' ' 

L-l 

— 0{n 3 ) and the rest of the proof is now pure 



This leads to (^1 + 
routine. 



I 



Lemma 4.8 If L < n, and m — ^ < —n^ 3k or k even and m — ^ > sk 
then 



k\L 



+ 



2rn\k\ 



0(1). 
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Proof: From (||) and from the mean value theorem we get 



ln(l + (l-^f) 



< 



n 



2 m 
n 



~^ 9" 



1 + i/„ 



2m\k 



with j/n^m^fc lying between and (1 ^ 

2m -4(fc-2) 

Under the lemma's assumptions we get (1 ) > 2 n 

n 

Thus, (BD gives: — 7^—^ > 2 n sk 



-2 



which shows that there exits Cfe such that for sufficiently large n 

'^^^^j^ > CkU-'^^^ > 0. As (1 - > O we get z/„,™,fc > and with L < n, 



the first inequality leads to: 



1 



a) 



(i + (i-^)''0 



< exp (^Ak 



Bk 
n 



As mentioned above, the proof of Claims [4.4| and 15 will follow from well- 
known results on the behavior of the binomial coefficients and on the distribution 
of the binomial law. 



Proof of Claim |4.4|: 

n 

As 2-" 



m=0 



1, Lemma (4.7) implies that: 



I]i(L,A;)-2-" 

\m — ^ \<n 3 

But, fc > 3 hence 1 — > ^ and De Moivre-Laplace's theorem asserts that: 



lim 2-" 

n— >+oo 



E 



2m \k 



< 



Proof of Claim 4.5; 

On the one hand, when k is odd and ni~ ^ > n^^^k^ —1 < 1 + (1 

2~'^n"3-. Similar computations as in the proof of Lemma |4.8| show that for 
sumciently large n, —1 < — -p^ — < 0, thus < (1 H -p^ — j < 1. 

\k) \k) 

From Dc Moivre-Laplace's theorem we know that for any k > 3 : 



lim 2" 



E 



\m-^\>n 3k 



^ 0. 
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Therefore, when k is odd, following Lemma 



On the other hand, when k is even Lemma 4.8 shows that 

|m— ■|^|>n^ 3r 

By parity and symmetry of binomial coefficients 

m— ■y>n3/c m— ■j<— nSfc 

Therefore, when k is even 



Therefore, Claim 4.5 is proved as soon as we are able to prove that for any 
positive real c < 9k ■ 



From Stirling's formula (see |^), we have for any m > 1, 

■ \ 1 / \ 1/2 / 



(4) 



, < ^ , " J exp 
TO/ v27r \m[n — m) J \ n 



thus. 

' n 
m 



O exp —n H { — ) 



Therefore, 

Tc,„-0( Y exp(-n5,,fc(^))) (5) 

I 4 

where 

= ln(l + (1 - 2xf){fk{x) - c). 

Now let us first note that for any c < 9k, {fk{x) ~ c) > 9k ~ c > 0. Hence, for 
any a > there exists 6 > such that: 

9c,k{x) > 6 when x £ [0, 1/2 — a]. 
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Then, the Taylor expansion gives for any fc > 3 : f/c,fe(l/2 — t) = + o{t'^). 
Thus, there exits an absolute positive constant Ck such that 

gc.ki^) > CkU-^/^'' when a; e [1/2 - a, 1/2 - n^^/^'^]. 

This proves the existence of some K > such that when m ~ n/2 < —n^~'^/'^^, 

exp(-n gc,k (— )) < exp(-ii'n^/^), 
with (15) , this establishes (0) and thus Claim kt.Sl. I 



5 An upper bound for the threshold 

Let Q denote the property is of maximal rank" then Pr{Q) = P(l)- 

Since P(i) + . . . + P(o) = 1, we have ^ 2--^P(,) < P^^) + ^ 



r=0 



Therefore, from Theorem 3.1 we get: 

1 + 



PiL) < P,uL(fc-XOR-SAT) < 
Since we know that fc-XOR-SAT exhibits a sharp threshold (see Theorem 



2.1 ) we get the second key fact: 



Proposition 5.1 The decreasing property Q exhibits a sharp threshold whose 
location coincides exactly with the one of k-XOR-SAT. Moreover if for some c 
there exist e > such that for L — on and all ( sufficiently large ) n 

P(L) < 1 — e then [3k < c. 

In other words, c is an upper bound for the threshold of fc-XOR-SAT as soon 
as we are able to prove that a matrix from Mcn,n,k is not of maximal rank with 
positive probability. 

When fc = 3, such a bound can be derived from the following result: 
Theorem 5.2 

„ , exp(— 3c) -t- 3ccxp(— 6c) — ccxp(— 9c) — 1 -I- c , , 

P(L) < 1 + o 1 . 

c 

Therefore, solving the equation 

exp(— 3c) + 3cexp(— 6c) — cexp(— 9c) — 1 + c = 0, 
in using MAPLE provides 

Ps < 0.9278. (6) 



10 



Proof: The matrices we consider, in Ml,„^3, have three units by row. Let 
A = (ttj ,) i=i,'-,L be such a matrix. For each row {an, . . . ,ai „), i = 1, . . . , L, 

' 3 = 1, --.n ' ' 

of A there are three distinct indices 1 < ii < 12 < is ^ n such that aij = 1 if 
j = ii,i2 or is, and otherwise; let Ii{A) denote the set {11,12, h}- So, to each 
row we can associate its 3- weight, Wi{A) defined as the following multiset: 

L L L 

fe=l fe=l fe=l 

Now, let us introduce three random variables T, U and V. 

For any random matrix A, let T{A) count the number of all-zero columns in 
^.Observe that these columns do not contribute to the rank of A. 
Let U{A) be defined by 

;7(^) = #{«/T4^,(A) = {1,1,1}}. 

Observe that each row contributing to U{A) underlines three columns such that 
only one of them contributes to the rank of A. 
Finally, let V{A) be defined by 

V{A) = #{i I Wi{A) = {1, 1, a} with a > 2}. 

Observe that each row contributing to V{A) underlines three columns such that 
two of them contribute to the rank of A. 
Thus, we have 

Rank{A) < Min{L,n-T{A) - 2U{A) - V{A)). 

Hence, 

if T + 2U + V>n—L then A is not of maximal rank, L. 

Therefore, 

P(L)<Pr{T + 2U + V<n-L). (7) 

Observe that 

n 

E{T + 2U + V) = Y, P^{T + 2U + V>i). 

Thus, for any integer e < n, 

E{T + 2U + V)<e+{n- e)Pr{T + 2U + V>e + 1). 

Hence, 

PriT + 2U + V>e + l)>^^^±^^L±n^. 

n — e 
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Thus (0) gives for L = cn 



E{T+2U+V) 



P(L) < 1 - 

Let us estimate E{T + 2U + V) = E{T) + 2E{U) + E{V). 
The random variables T and U are the sum of indicator variables: 

n 

where Tj{A) = 1 iff the jth column of A is all-zero; 

l<ii <i2 <i3<n 

where Ui-^i^i^^A) = 1 iff there exists i such that Ii{A) = {ii,Z2,*3}, and 
={l,'l,l}. 

Now, Pr(T, = 1) = (^-^^^ ' ^ence 

£;(T) > nexp(-3c)(l - o(l)) (8) 

and 

P'r{Uiui2M) = P — TITl — ' therefore, 
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(3) 

S(?7) > cnexp(-9c)(l-o(l)). (9) 
Finally, let us estimate E{V). In the same way 



where Vi^^i^iA) = 1 iff there exists i such that {ii,i2} C Ii{A), J2k=i = ^' 
ELi = 1, and W,{A) = {1, 1, with a > 2. 

Let us introduce the random variables Zi^^i^{A) such that Zi^,i^{A) = 1 iff there 
exists i such that {11,12} C Ii{A), X]fc=i Ofc.ii = 1 and X]fc=i «fc,i2 = 1- 



Pr{Z,,^,, = 1) = = 1 and F,,.,, = 1) + = 1 and F,,,,, = 0) 

= Pr{V,,,,,^l)+ PriU^„^2.^s ^ ^) 

isfih ,12} 

= E{Vn.2) + in-2)Pr{U,,^,,^,,^l). 

But, 



fn-2\L-i 

E{Z,,,,,) = L{n~2)- 



(3) 



12 



Therefore, 

(V)^'^-(V) 



E{V,,,,,)^L{n~2)- 



\3J 

And finally, 

E(y) > (3cnexp(-6c) - 3cnexp(-9c))(l - o(l)). (10) 

Finally, Theorem p| follows from (|), (|) and @. I 

The proof of Theorem ^.2| relies on a construction which can be carried out 
for all integer fc > 3. For more readability we have restricted our attention to 
the generic case k — 3, however with similar considerations one can obtain the 
following general result : 

Theorem 5.3 Jork>A: 

D /I exp(-c/c) + cufe(c) + Ufe(c) - 1 + c 

P(L) < 1 ^o(l), 

^ ' c 

where 

Uk{c) = exp(-cfc(fc - jmk ^j + l)-{k-j-2){k~j), 

Vk{c) = 3 exp(-2cfc) + {-k^ + 3fc - 1) exp(-cfc2). 

Therefore, solving the equation 

exp(— cfc) + cufe(c) + Wfe(c) — 1 + c = 0, 
in using MAPLE provides 

/34 < 0.9721, /Js < 0.9914, /Jg < 0.9971 (11) 



6 Experiments 

In this section, focussing on the generic case 3-XOR-SAT, we supplement the 
preceding rigorous results with experimental results. In the previous sections, 
we have determined an approximate scaling of a window in which a phase tran- 
sition of the satisfiability must be observed, namely for a ratio number L of 
clauses to number n of variables below 0.8894, the probability of satisfiability 
tends to 1 as L and n tend to infinity and for a ratio above 0.9278, that prob- 
ability tends to 0. In order to illustrate the phase transition and to estimate 
empirically the location of the critical value C3 of the ratio for which the transi- 
tion occurs, we have made experiments consisting in generating at random (in 
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Figure 1: 



drawing uniformly and independently) 3-XOR-SAT formulas over 100, 200, 300 
and 400 variables with a ratio varying from 0.70 to 1.14 in steps of 0.1. For 
each of these values of ratio, a sample of 1000 formulas has been solved with a 
computer program. The proportion of satisfiable formulas for each considered 
value of ratio has been plotted on the above Figure 1. It can be seen that the 
four smooth lines connecting the consecutive points corresponding to 100, 200, 
300 and 400 variables, straighten as the number of variables increases showing 
thus strong empirical evidence of the sharp phase transition proved in 0| . The 
crossing of these lines suggest that the critical value C3 of the transition is a 
little lower than 0.92 for 3-XOR-SAT. 



7 Conclusion 

The fc-XOR-SAT problem is polynomial time solvable. Compared to the studies 
carried out on the phase transition of the SAT problem, this gives hope to get 
here an easier study. In a first step we have made precise the link between the 
fc-XOR-SAT's phase transition and the rank of sparse random Boolean matrices. 
This last problem has been extensively studied by Russian mathematicians (see 
[ pi] and jl^), but reveals hard combinatorial and probabilistic problems. Our 
approach, which consists in using the sharpness of the threshold in order to 
specify its location, enables us to get good and interesting bounds for the critical 
ratio of A:-XOR-SAT's phase transition. Indeed, experiments show the tightness 



of the upper bound. Moreover let us emphasize that Theorem 4.2 provides a 
new proof of Kolchin and Khokhlov's results , as well as a simpler expression 
for the corresponding critical value. 

In conclusion, our work illustrates the importance of directing a lot of work 
towards obtaining general conditions for sharpness of a phase transition. 
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